Use of Self-Healing Techniques for Highly-Available Distributed Monitoring

The paper addresses the self-healing aspects of the monitoring systems. Nowadays, when the complex distributed systems are concerned, the monitoring system should become "intelligent" - as the first step it can guide the user what should be monitored. The next level of the "intelligence" can be described by the term "self-healing". The goal is to provide the capability that a decision made automatically by the monitoring system should force the system under monitoring to behave more stable, reliable and predictable. In the paper a new monitoring system is presented: AgeMon is an agent based, distributed monitoring system with strictly defined roles which can be performed by the agents. In the paper we discuss self-healing in the context of monitoring. When the self-healing of the monitoring system is concerned, a good example is the case where it is possible to lose the monitoring data due to the storage problems. AgeMon can handle such problems and automatically elects substitute persistence agents to store the data

SCALING EVOLUTIONARY PROGRAMMING WITH THE USE OF APACHE SPARK

Organizations across the globe gather more and more data, encouraged by easy-to-use and cheap cloud storage services. Large datasets require new approaches to analysis and processing, which include methods based on machine learning. In particular, symbolic regression can provide many useful insights. Unfortunately, due to high resource requirements, use of this method for large-scale dataset analysis might be unfeasible. In this paper, we analyze a bottleneck in the open-source implementation of this method we call hubert. We identify that the evaluation of individuals is the most costly operation. As a solution to this problem, we propose a new evaluation service based on the Apache Spark framework, which attempts to speed up computations by executing them in a distributed manner on a cluster of machines. We analyze the performance of the service by comparing the evaluation execution time of a number of samples with the use of both implementations. Finally, we draw conclusions and outline plans for further research

Evolution-by-Coevolution of Neural Networks for Audio Classification

Neural networks are increasingly used in recognition problems, including static and moving images, sounds, etc. Unfortunately, the selection of optimal neural network architecture for a specific recognition problem is a difficult task, which often has an experimental nature. In this paper we present the use of evolutionary algorithms to obtain optimal architectures of neural networks used for audio sample classification. We extend the Pytorch DNN Evolution tool implementing co-evolutionary algorithms which create groups of neural networks that solve a given problem with a certain accuracy, with the support for problems in which training data consists of audio samples. In this paper we use the co-evolutionary approach to solve a sample sound classification problem. We describe how the sound data was prepared for processing with the use of the Mel Frequency Cepstral Coefficients (MFCC). Next we present the results of experiments conducted with the AudioMnist dataset. The obtained neural network architectures, whose classification accuracy is comparable to the classification accuracy attained by the AlexNet neural network, and their implications are discussed

INTEROPERABILITY OF MONITORING-RELATED TOOLS

Networking, distributed and grid computing have become the commonly used paradigms ofprogramming. Due to the complicated nature of distributed and grid systems and the increasing complexity of the applications designed for these architectures, the development processneeds to be supported by different kinds of tools at every stage of a development process. Inorder to avoid improper influences of one tool to another these tools must cooperate. Thecooperation ability is called interoperability. Tools can interoperate on different levels, fromexchanging the data in common format, to a semantical level by executing some action asa result of an event in another tool. In this paper we present some interoperability models,with focus on their advantages and major problems due to their use. We also present aninteroperability model designed and used in the JINEXT extension to OMIS specification,intended to provide interoperability for OMIS-compliant tools

Semantic-Oriented Performance Monitoring of Distributed Applications

Monitoring services are an essential component of large-scale computing infrastructures due to providing information which can be used by humans as well as applications to closely follow the progress of computations, to evaluate the performance of ongoing computing, etc. However, the users are usually left alone with performance measurements as to the interpreting and detecting of execution flaws. In this paper we present an approach to the performance monitoring of distributed applications based on semantic information about the monitored objects involved in the application execution. This allows to automate the guidance on what to measure further to come to a source of performance flaws as well to enable reacting on interesting events, e.g. on exceeding SLA parameters. Our research comprises the implementation of a robust system with semantics, which is not biased to an underlying ``physical'' monitoring system, giving the end user the power of intelligent monitoring functionality as well as the independence of the heterogeneity of distributed infrastructures

TOWARDS AUTONOMIC SEMANTIC-BASED MANAGEMENT OF DISTRIBUTED APPLICATIONS

In this paper we present our approach to the management of distributed systems basedon semantic description of available resources. We use ontologies for a semantic descriptionof the monitored system and other aspects of monitoring and management (such as metrics)and introduce a feedback loop on underlying infrastructure. Such an approach allows toautomate monitoring and the ease the work of administrator. We introduce concepts behinda novel automatic management system, SAMM, developed within our research. We discussthe core mechanisms used in the system – the estimation of future measurements, approachto knowledge gathering, and the process of decision making. Then we provide some detailson the architecture and implementation of SAMM

Agent-Based Monitoring Using Fuzzy Logic and Rules

In this paper we present two solutions of monitoring automation for distributed systems. We develop this system to automate monitoring of distributes systems. Both solutions are aimed to monitor data storage and web services like web page servers. The first solution implemented in a system called Saude-Net, is an rule-based top level monitoring tool. In this system there are implemented rules which provide conditions which refer to one or more measured values. This system is able to choose the best action for an observed situation, e.g. a failure. It is possible to define more than one rule which relate to the same monitoring resource. The second concept presented in this paper refers to a fuzzy logic agent based approach to network monitoring. It is called SAMM compliant Agent. It is an extension to the Semantic-based Autonomous Monitoring and Management system (SAMM). On the one hand, it uses rules to define simple actions, based on a simple condition and an action description. On the other hand the main knowledge of this solution is defined by fuzzy logic. This system is able to manage and modify its knowledge to better fit to monitored resources. The knowledge in this concept is distributed among all the agents. The agents residing on a different hosts handle their parts of the knowledge and are capable to share/exchange them

Agent-based hierarchical approach for executing bag-of-tasks in clouds

Numerous unrelated, independent (no inter-task communication) tasks called “bag-oftasks”(BoTs) compared with message passing applications can be highly parallelised andexecuted in any acceptable order. A common practice when executing bag-of-tasks applications(BoT) is to exploit the master-slave topology. Cloud environments offer some featuresthat facilitate executing BoT applications. One of the approaches to control cloud resourcesis to use agents that can flexibly act in a dynamic environment. Given these assumptions wedesigned a combination of these approaches, which can be classified as: a distributed, hierarchicalsolution to the issue of scalable executing of bag-of-tasks. The concept of our systemrelates to a project that is focused on processing huge quantities of data incoming from anetwork of sensors by the Internet. Our aim is to create a mechanism for processing such dataas a system which executes jobs while exploiting load balancing for cloud resources using,e.g., Eucalyptus. The idea is to create a hybrid architecture which takes advantage of somecentralized parts of the system and full distributedness in other parts. On the other handwe balance dependencies between the system components using a hierarchic master-slavestructure